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COMPUTERIZED ADAPTIVE IMAGING 


FIELD OF THE INVENTION 
The present invention relates to imaging generally and more particularly to post 
acquisition image processing generally. 

BACKGROUND OF THE INVENTION 
Post acquisition image processing is well known in the literature. Publications which 
describe the general state of the art in post acquisition image processing are : Lim, J.S. "Two 
dimensional signal and image processing". Englewood Cliffs, NJ. Prentice Hall. (1990); Russ, J.C. 
"Image processing handbook". CRC Press. (1992); Pratt, W.K. "Digital image processing". NY John 
Wiley & Sons, Inc. (1991); Rosenfeld, A. and Kak, A.C. "Digital picture processing". Academic 
Press. (1976); Castleman, K.R. "Digital Image Processing". Prentice-Hall Inc. Englewood Cliffs, 
New Jersey. (1979). 

Imaging which provides information relating to refractive characteristics in a imaged 
volume is known for extremely limited applications. In microscopy, Smith, Nomarski and Differential 
Interference Contract (DIC) imaging is known and is described in the following publications: 
Nomarski, G. "Microinterferometre differential a ondes polarisees". J. Phys. Radium 16:9S-11S 
(1955); Lang, W. "Differential-Interferenz-Miikroskopie". Carl-Zeiss, Oberkochen (1975); Inoui, S. 
and Spring, K.S. "Video Microscopy: the fundamentals". 2nd edition. Plenum Press, NY. (1997). 
Tanford, C. "Physical chemistry of macromolecules". John Wiley NY. (1961). Appendix C describes 
-classical R a vl e foh inf p rferpnre mPthnH c j P h ilpnt < y > nS nn mrthnds hrnpd nn *rHWn imnm* 


Lamm method of line displacement, and Gouy interference method all developed for determination 
of one dimensional refractive index variations. 
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Computer a^Mis of DIC imaging is not readily a^Med. Known instances' are 
described in the following publications: Allen, R.D., Allen, N.S. and Travis, J.L. "Video-enhanced 
contrast, differential interference contrast (AVEC-DIC) microscopy: a new method capable of 
analyzing microtubule related motility in the reticulopodial network of Allogromia laticollaria." Cell 
Motility 1: 291-302 (1981); Cogswell, C.J. and Sheppard, CJ.R. "Confocal differential interference 
contrast (DIC) microscopy: including a theoretical analysis of conventional and confocal DIC 
imaging". J. Microsc. 165:81-101 (1992); Gelles, J., Schnapp, BJ. and Sheetz, M.P. "Tracking 
kinesin-driven movements with nanometre-scale precision", Nature 331:450-453 (1988); Hdusler, G. 
and Kvrner, E. "Imaging with expanded depth of focus". Zeiss Inform. 29: 9-13 (1987); Preza, C, 
Snyder, D.L. and Conchello, J-A. "Image reconstruction for three-dimensional transmitted light DIC 
microscopy". SPIE 2984:220-231 (1997);.Schormann, T. and Jovin, T.M. "Contrast enhancement 
and depth perception in three-dimensional representations of differential interference contrast and 
confocal scanning laser microscope images". J. Microsc. 166:155-168 (1992). 

Computerized ray tracing between discrete refractive and reflective surfaces is 
extremely well developed, but is not well known in the environment of non-homogeneous indices of 
refraction. This area is described in the following publications: Hecht E. and Zajac A. "Optics" 2nd 
ed. Addison-Wesley Reading MA (1997); Jenkins, F.A, and White, H.E. "Fundamentals of optics". 
McGraw-Hill, NY (1950) ch.8: Ray Tracing. 

Calculation of point spread functions (PSF) is extremely well known as described in 
the following publication: Born M. and Wolf E. "Principles of Optics" Pergamon London 
(1959);Goodman J.W. "Statistical Optics" John Wiley & Sons NY (1985);Hecht E. and Zajac A. 
"Optics" 2nd ed. Addison-Wesley Reading MA. (1997); Gibson S.F. and Lanni F. "Diffraction by 
sigeaiaf =-aperturc as a model for three-dimensional optical microscopy". Opt. Soc. Am, A_ 


6:1357-1367 (1989); Gibson S.F. and Lanni F. "Modeling aberrations due to mismatched layers for 
3-D microscopy" SPIE optics in complex systems 1319:470-471 (1990); Gibson S.F. and Lanni F. 
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"Experimental test of an analytical model of aberration in an oil-immersion objective lens used in 
three-dimensional light microscopy". J. Opt. Soc. Am. A 8:1601-1613 (1991). 

Deconvolution of three dimensional microscopic images having location independent 
PSF is well known and is described in the following publications, some of them authored by some of 
the present inventors: Jansson, P.A. ed. "Deconvolution of images and spectra". Academic Press NY 
(1997); Agard, D.A. and Sedat, J.W.. "Three-dimensional architecture of a polytene nucleus". 
Nature 302:676-681 (1984); Agard, D.A., Hiraoka, Y, Shaw, P. and Sedat, JW. "Fluorescence 
microscopy in three dimensions". Methods in Cell Biology 30: 353-377 (1989); Castleman, K.R. 
"Digital Image Processing". Prentice-Hall Inc. Englewood Cliffs, New Jersey (1979). Correction of 
telescopic images by the use of suitably distorted mirrors and deconvolution of two dimensional 
telescope images having location dependent PSF are described in the following publications: Boden, 
A.F., Reeding, D.C, Hanisch, R.J., Mo, J. and White, R. "Comparative results with massively parallel 
spatially-variant maximum likelihood image restoration". Bui Am Astr. Soc 27:924-929 (1995); 
Boden, A.F., Reeding, D.C, Hanisch, RJ. and Mo, J. "Massively parallel spatially-variant maximum 
likelihood restoration of Hubble space telescope imagery". J Opt Soc Am A 13: 1537-1545 (1996); 
Jansson, P.A. ed. "Deconvolution of images and spectra". Academic Press NY (1997); Tyson R.K. 
"Principles of Adaptive Optics" Academic Press NY (1991). Reconstruction of blurred images from 
point objects is described in the following publications: Carrington, W.A., Lynch, R.M., Moore, 
D.W., Isenberg, G., Fogarty, K.E. and Fay, F.S. "Superresolution three-dimensional images of 
fluorescence in cells with minimal light exposure". Science 268:1483-1487 (1995); Femino, A.M., 
Fay, F.S., Fogarty, K., and Singer, R.H. "Visualization of single RNA transcripts in situ". Science 
280:585-590 (1998). 

SUMMARY OF THE INVENTION 
The present invention seeks to provide improved apparatus and techniques for post 
acquisition image processing. 


There is thus^iCvided in accordance with a preferrec^nbodiment of the present 
invention apparatus for computational adaptive imaging including an image information acquirer 
providing information relating to the refractive characteristics in a three-dimensional imaged volume, 
a ray tracer, utilizing the information relating to the refractive characteristics to trace a multiplicity of 
rays from a multiplicity of locations in the three-dimensional imaged volume through the 
three-dimensional imaged volume, thereby providing a location dependent point spread function and 
a deconvolve^ utilizing the location dependent point spread function, to provide an output image 
corrected for distortions due to variations in the refractive characteristics in the three-dimensional 
imaged volume. 

Preferably, the image information acquirer acquires at least two three-dimensional 
images of a three-dimensional imaged volume, at least one of the two three-dimensional images 
containing the information relating to the refractive characteristics in a three-dimensional imaged 
volume. 

In accordance with a preferred embodiment of the present invention, the image 
acquirer acquires at least three three-dimensional images of the three-dimensional imaged volume. 

Preferably, the image acquirer acquires a plurality of three-dimensional images of the 
three-dimensional imaged volume, each the image having a discrete wavelength band. 

Alternatively, the image acquirer acquires a multiplicity of three-dimensional images 
of the three-dimensional imaged volume, each the image having a wavelength band which is part of a 
continuum represented by the wavelength bands of the multiplicity of three-dimensional images. 

In accordance with a preferred embodiment of the present invention, the ray tracer 
and the deconvolver utilize the information relating to the refractive characteristics in a 
-ttree=dimensional imaged volume obta fr red Qum one of t he t hr ee- dimensional images to correct_at_ 


least another one of the three-dimensional images. 

According to one embodiment of the present invention, the three-dimensional images 
are electromagnetic energy images. Preferably, the three-dimensional images are infrared images. 
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Alternatively, the three-dimensional images are nor>electromagnetic images. 
In accordance with a preferred embodiment of the present invention, the image 
acquirer receives digital image data from a digital image source and derives therefrom the 
information relating to the refractive characteristics in a three-dimensional imaged volume. 

Preferably, the ray tracer and the deconvolver operate repeatedly over time to provide 
a multiplicity of output images, each corrected for distortions due to variations in the refractive 
characteristics in the three-dimensional imaged volume. 

In accordance with one embodiment of the present invention, the output image is an 
acoustic image and the refractive characteristics are characteristics of a material which the passage of 
acoustic energy therethrough. 

In accordance with an alternative embodiment of the present invention, the output 
image is an electromagnetic image and the refractive characteristics are characteristics of a material 
which the passage of electromagnetic energy therethrough. 

There is also provided in accordance with a preferred embodiment of the present 
invention a method for computational adaptive imaging including the steps of: 

providing information relating to the refractive characteristics in a three-dimensional 
imaged volume; 

ray tracing, utilizing the information relating to the refractive characteristics, a 
multiplicity of rays from a multiplicity of locations in the three-dimensional imaged volume through 
the three-dimensional imaged volume, thereby providing a location dependent point spread function; 
and 

deconvolving, utilizing the location dependent point spread function, thereby 
providing an output image corrected for distortions due to variations in the refractive characteristics 
in the three-dimensional imaged volume. 

Preferably, the step of providing information includes acquiring at least two 
three-dimensional images of a three-dimensional imaged volume, at least one of the two 
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three-dimensional images ^^wning the information relating to the^^active characteristics in a 
three-dimensional imaged volume. 

In accordance with a preferred embodiment of the present invention, the step of 
providing information includes acquiring at least three three-dimensional images of a 
three-dimensional imaged volume. 

Preferably, the step of providing information includes acquiring a plurality of 
three-dimensional images of the three-dimensional imaged volume, each image having a discrete 
wavelength band. 

In accordance with a preferred embodiment of the present invention, the step of 
providing information includes acquiring a multiplicity of three-dimensional images of the 
three-dimensional imaged volume, each the image having a wavelength band which is part of a 
continuum represented by the wavelength bands of the multiplicity of three-dimensional images. 

In accordance with one embodiment of the present invention, the three-dimensional 
images are electromagnetic energy images. Preferably, the three-dimensional images are infrared 
images. 

In accordance with an alternative embodiment of the present invention, the 
three-dimensional images are non-electromagnetic images. 

Preferably, the step of providing includes receiving digital image data from a digital 
image source and deriving therefrom the information relating to the refractive characteristics along a 
multiplicity of light . paths in a three-dimensional imaged volume. 

In accordance with a preferred embodiment of the present invention, the steps of 
providing information, ray tracing and deconvoluting operate repeatedly over time to provide a 
ra&pfe& y of output imagesr-e ash= eorrected for distortions due to variations in the=refraetive= 
characteristics in the three-dimensional imaged volume. 

There is also provided in accordance with a preferred embodiment of the present 
invention apparatus for utilizing differential interference contrast images to provide 
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three-dimensional refractive index information including a line integrator operating on differential 
interference contrast images displaying a directional derivative of refractive index of an object to 
invert the directional derivative thereof, thereby providing a plurality of two-dimensional 
representations of the refractive index of the object. 

There is additionally provided in accordance with a preferred embodiment of the 
present invention apparatus for utilizing differential interference contrast images to provide 
three-dimensional refractive index information and also including a deconvolver performing 
deconvolution of the plurality of two-dimensional representations of the refractive index of the 
object, thereby reducing out-of-focus contributions to the two-dimensional representations of the 
refractive index of the object. 

There is further provided in accordance with a preferred embodiment of the present 
invention a method for utilizing differential interference contrast images to provide three-dimensional 
refractive index information including performing line integration on differential interference contrast 
images displaying a directional derivative of refractive index of an object to invert the directional 
derivative thereof, thereby providing a plurality of two-dimensional representations of the refractive 
index of the object. 

Additionally in accordance with a preferred embodiment of the present invention 
there is provided a method for utilizing differential interference contrast images to provide 
three-dimensional refractive index information and also including performing deconvolution of the 
plurality of two-dimensional representations of the refractive index of the object, thereby reducing 
out-of-focus contributions to the two-dimensional representations of the refractive index of the 
object. 

Fu rther in ac c or d a nce with a preferred embodiment of the present invention the rgjs^ 
provided apparatus for ray tracing through a medium having multiple variations in refractive index 
including: 
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a computerBMploying an analytically determined of a ray through the 

multiplicity of locations in the medium, for a plurality of rays impinging thereon in different 
directions, by utilizing known local variation of the refractive index at a multiplicity of locations in 
the medium. 

There is additionally provided in accordance with a preferred embodiment of the 
present invention a method of ray tracing through a medium having multiple variations in refractive 
index including: 

determining local variation of the refractive index at a multiplicity of locations in the 

medium; 

analytically determining the path of a ray through the multiplicity of locations in the 
medium, for a plurality of rays impinging thereon in different directions. 

Still further in accordance with a preferred embodiment of the present invention there 
is provided apparatus for confocal microscopy including: 

a ray tracer, employing known variations of the refractive index in a 
three-dimensional sample for determining the paths of a multiplicity of rays emerging from at least 
one point in the sample and passing through the sample, thereby determining an aberrated wavefront 
for each the point; and 

an adaptive optics controller utilizing the aberrated wavefront to control an adaptive 
optical element in a confocal microscope, thereby to correct aberrations resulting from the variations 
in the refractive index. 

There is additionally provided in accordance with a preferred embodiment of the 
present invention a method for confocal microscopy including: 

dctcnrin ifig= yariatiQfls of the refractive index in a three-dimensional sample; 


determining the paths of a multiplicity of rays emerging from at least one point in the 
sample and passing through the sample, thereby determining an aberrated wavefront for each the 
point; and 
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utilizing the aberrated wavefront to control an adaptive optical element in a confocal 
microscope, thereby to correct aberrations resulting from the variations in the refractive index. 


BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more fully from the 
following detailed description, taken in conjunction with the drawings in which: 

Figs. 1A and IB are respectively representations of three dimensional microscopic 
images of an object according to the prior art and in accordance with the present invention; 

Fig. 2 is a simplified partially pictorial, partially block diagram illustration of 
apparatus for computational adaptive imaging constructed and operative in accordance with a 
preferred embodiment of the present invention; 

Fig. 3 is a simplified flow chart of the operation of the apparatus for computational 
adaptive imaging of Fig. 2; 

Fig. 4 is a simplified diagram of line integration functionality which is carried out by 
the apparatus of Fig. 2; 

Fig. 5 is a simplified diagram of ray tracing functionality which is carried out by the 
apparatus of Fig. 2; 

Fig. 6 is a simplified diagram of wavefront integration functionality which is carried 
out by the apparatus of Fig. 2; and 

Fig. 7 is a simplified flow chart illustrating deconvolution functionality which is 
carried out by the apparatus of Fig. 2. 


^ BRIEF^DES € RIPTION O F APPENDICES 


Appendix A is a hexadecimal listing of a program which converts DIG images to a 
three dimensional map of refraction indices of a corresponding imaged object by line integration; 


9 


Appendix B i^Bfexadecimal listing of 



a program 


whic 1 



tlates location dependent 


PSF by ray tracing through the imaged object, taking into account the three dimensional map of 
refraction indices thereof, calculating a distorted wavefront and computing therefrom a diffraction 
integral, thereby to provide the PSF; 

Appendix C is a hexadecimal listing of a program which deconvolutes the location 
dependent PSF from a distorted image, thus providing a corrected image of the imaged object; and 

Appendix D is a collection of two not-yet-published articles describing preferred 
embodiments of the present invention. 


three dimensional microscopic images of an object according to the prior art and in accordance with 
the present invention. 


an ideal sample displaying no substantial variation in refractive index. When an ideal optical system is 
focussed at slightly different depths in the sample, a three dimensional image appears, including a 
plurality of two-dimensional diffraction limited images designated by Roman numerals I, n, III, IV 
and V in Fig. 1A. The prior art, described, inter alia in the aforesaid Agard et al 1989 reference is 
capable of deconvolving the plurality of two-dimensional images I - V to a deconvoluted three 
dimensional image, including a single in focus two-dimensional image VI, which contains 
substantially all of the image intensity from images I - V and a plurality of out of focus 
two-dimensional images which substantially contain no image intensity. 

Thft p rior art, however— does not know how to deal a d e qu a t e l y with sa m ple whtrh 
display substantial variation in refractive index. 

The present invention provides a solution to this problem, as illustrated in Fig. IB 
which shows images of a non-ideal sample, such as a biological sample, displaying substantial 
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DETAILED DESCRIPTION OF THE INVENTION 


Reference is now made to Figs. 1A and IB, which are respectively representations of 


It is known in the prior art to image a point source or a collection of point sources in 


vanation in refractive index. When an ideal optical system is focussed at slightly different depths in 
the non-ideal sample, a three-dimensional image appears, including a plurality of two-dimensional 
diffraction limited images designated by Roman numerals I, n, m, IV and V in Fig. IB. In contrast 
to the images appearing in Fig. 1 A, it is seen that these images are extremely non-uniformly distorted 
due to variations in the refractive index at various locations in the sample. 

The present invention, as will be described hereinbelow, is capable of deconvolving 
the plurality of two-dimensional images I - V in Fig. IB, notwithstanding their great distortions due 
to variations in the refractive index at various locations in the sample, to a substantially 
non-distorted, deconvoluted three-dimensional image, including a single in-focus two-dimensional 
image VI, which contains substantially all of the image intensity from images I - V and a plurality of 
out-of- focus two-dimensional images which substantially contain no image intensity. 

Reference is now made to Fig. 2, which is a simplified partially pictorial, partially 
block diagram illustration of apparatus for computational adaptive imaging constructed and 
operative in accordance with a preferred embodiment of the present invention. The apparatus of Fig. 
2 typically includes a conventional computer-controlled optical microscope 10, such as a microscope 
manufactured by Carl Zeiss and sold under the name Axioskop. Associated with microscope 10 there 
are preferably provided computer controlled filter wheels 12 and 14. Filter wheel 12 is disposed 
downstream of an excitation light source 16, such as a mercury arc lamp, which provides 
epi-illumination via a computer-controlled shutter 18 and filter wheel 12 to a dichroic mirror 20 
within the microscope 10. 

Trans-illumination is provided to a sample 22, typically mounted between two slides 
24 and 26, by means of a halogen light source 28 via a computer-controlled shutter 30. Depending 
"^^vliHT or s l ii ilf ^ n Iff n inl 10 is^pf^T^h^-nfltmp l n r c mycn trnm i l l um ination or cpi illumination so 
as to provide either a DIC image or a fluorescent image respectively. It is noted that DIC images 
conventionally require additional optical elements in the imaging light path, which are omitted from 
Fig. 2 for the sake of conciseness and clarity. 
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In accordan<!^^^th a preferred embodiment of the pj^^it invention, the DIC or 
fluorescent image is received via computer-controlled filter wheel 14 by a digital CCD sensor 32, 
such as a Photometries PXL. Preferably a polarizer is included in one of the windows of filter wheel 
14 and is employed for DIC imaging. 

In accordance with a preferred embodiment of the present invention, the output of the 
digital CCD sensor 32 is supplied to a computer 34, such as a Silicon Graphics 02 workstation which 
performs the following functionality: 

ray tracing, utilizing information relating to the refractive characteristics in the 
sample, a multiplicity of rays from a multiplicity of locations in the sample through the sample, 
thereby providing a location dependent point spread function; and 

deconvolving, utilizing the location dependent point spread function, thereby 
providing an output image corrected for distortions due to variations in the refractive characteristics 
in the three-dimensional imaged volume. 

Preferably wavefront integration is performed following the ray tracing and prior to 
the deconvoluting. 

In accordance with a preferred embodiment of the present invention, the information 
relating to the refractive characteristics of the sample is provided by line integration of the DIC 
image. 

Reference is made to Fig. 3, which is a simplified flow chart of the operation of the 
apparatus for computational adaptive imaging of Fig. 2. As illustrated in Fig. 3, three dimensional 
images from the microscope 10 and CCD 32, typically include a plurality of observed fluorescent 3D 
images 40 and a differential interference contrast (DIC) 3D image 42. 

I n arrp f^^ with a- prpferrfid e m bodim ent of the pr es e nt inven t i on , l i ne in teg ration 
44 is performed on the DIC image, preferably using the software exemplified in Appendix A. The 
line integration preferably provides a 3D refractive index map 46. 
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In accordance with a preferred embodiment of the present invention, ray tracing 48 is 
performed on the 3D refractive index map 46 and the resulting wavefront is subject to wavefront 
integration 50, thus providing a location dependent point spread function (PSF) 52. Preferably the 
ray tracing 48 and the wavefront integration 50 is performed by using the software exemplified in 
Appendix B. 

In accordance with a preferred embodiment of the present invention, the location 
dependent PSF is utilized in a deconvolution step 54 which is applied to the observed fluorescent 3D 
image 40. Preferably the deconvolution 54 is performed by using the software exemplified in 
Appendix C. 

In accordance a preferred embodiment of the invention, the deconvolution produces a 
corrected image, such as the image illustrated at reference VI in Fig. IB. 

Reference is now made to Fig. 4, which is a simplified diagram of line integration 
functionality which is carried out by the apparatus of Fig. 2, preferably utilizing the software 
exemplified in Appendix A. Fig. 4 illustrates a sample, such as a sphere 60 of uniform index of 
refraction, which differs from that of surrounding media 62. Sample 60 may be taken to represent a 
nucleus, while the surrounding media may be taken to represent cytoplasm. A scan line 64 illustrates 
the index of refraction as a function of location along line 65. The characteristic PSF of the 
microscope 10 operating in the DIC imaging mode is illustrated at reference numeral 66. The PSF is 
also illustrated by trace 68. 

The output image of sample 60 of the CCD sensor 32 operating in a DIC mode is 
illustrated at reference numeral 70 and also by trace 72. Line integration of the output image, 
preferably utilizing the software exemplified in Appendix A, produces an image 74 which accurately 
represents the original sample, seen at referenc e numerals 60 and 62 . 

It is appreciated that line integration of DIC images has two complex contributions in 
reality. One is the accumulation of noise in the integration and the second is the slow contribution of 
low-intensity out-of-focus contributions. The problem of noise is preferably alleviated by introducing 
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a decay function in the line^J^ration. This decay feature appears in^^endix A. The out-of-focus 
contribution can be reduced significantly by deconvoluting the integrated DIC images. This feature 
also appears in Appendix A 

Reference is now made to Fig. 5, which is a simplified diagram of ray tracing 
functionality which is carried out by the apparatus of Fig. 2. As seen in Fig. 5, it is seen that an 
isotropic fan of rays is drawn from every point in an imaged volume, e.g. the sample. These rays are 
ray traced, preferably using the software embodied in Appendix B, such that their paths and phases 
are modified in accordance with variations in the refractive index of the sample. The result of the ray 
tracing is an aberrated wavefront for every point in the imaged volume. 

Reference is now made to Fig. 6, which is a simplified diagram of prior art wavefront 
integration functionality which is carried out by the apparatus of Fig. 2 preferably using the software 
embodied in Appendix B. As seen in Fig. 6, for every point in the vicinity of a focus RO, an 
interference integral is taken over the wavefront. The integrations for all of the points in the vicinity 
of the focus RO define the three-dimensional point spread function, as described inter alia in the 
aforesaid prior art reference to Goodman. 

Reference is now made to Fig. 7, which is a simplified flow chart illustrating 
deconvolution functionality which is carried out by the apparatus of Fig. 2, preferably utilizing the 
software embodied in Appendix C. As seen in Fig. 7, the observed image 80, such as the observed 
fluorescent 3D image 40 (Fig. 3) is deconvoluted using the point spread function (PSF) calculated as 
described hereinabove with reference to Figs. 5 and 6. It is appreciated that the location dependent 
PSF is preferably a multi-resolution PSF. This feature appears in the software of Appendix C. This 
calculation is preferably performed in an iterative process which endeavors to provide progressively 
- bette r-approxifflation s of theimage d sampl e . — 


This iterative process typically proceeds by initially defining a guessed object 82, 
which typically is identical to the observed image. The guessed object is convoluted with a location 
dependent point spread function 84, such as that derived by ray tracing and wavefront integration 
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(Fig. 3) to produce a blurred image 86. This blurred image is compared with the obseived image and 
a correction based on the comparison is iteratively applied to the guessed object. Following multiple 
iterations, the guessed object constitutes the corrected 3D image of the sample in accordance with a 
preferred embodiment of the present invention. 

A preferred method for constructing a system operative in accordance with a 

preferred embodiment of the present invention is now described with reference to Appendices A, B, 

andC: 

a) Provide a SILICON GRAPHICS workstation, such as model 02, with 128 
megabytes of RAM memory running the IRIX Release 6.3 IP32 operating system; 

b) Install the DELTA VISION optical sectioning microscope system, commercially 
available from Applied Precision Inc., 1024 12th Ave NW, Issaquah WA 98027-8929, USA; 

c ) Create three binary files from the hexadecimal listings of Appendices A, B and C; 

d) Uncompress the three binary files using GZEP version 1 . L2 into three executable files 
named "pardic," "dictrace," and "decon_full"; 

e ) Provide at least one 3D DIC image and one 3D fluorescent image using the 
DELTA VISION system; 

f) Run the pardic executable file to perform the following: 

Read one optical section from the 3D DIC image; 

Use the T submenu to filter the image from slowly varying backgrounds; 
Use the line integration submenu TJ 1 to convert the 3D DIC image to a 3D 

refractive index map; 

For 'filling up' the effect of decay in the line integration use 
J3IC^d ec onvo 1ntion su bmenu W Tn Hn th^ J Fjrvt t ho r>TP p ™t ppr^ filiation using the 

kernel submenu K'; 
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For al^Hproved index map, read three conseci^^optical DIC sections and 
use the 'Z 1 submenu to subtract out-of-focus contributions using nearest neighbor approximation of 
the three-dimensional deconvolution; 

Output the refractive index 3D map section by section; 

g) Run the dictrace executable file to perform the following: 

Read in the refractive index 3D map. Simulated maps including different 
gradient functions are available as alternative inputs; 

Create a fan of rays originating from a point and homogeneously spanning a 
cone with a defined solid angle. The opening angle and angular increment can be defined. Typically 
200,000 rays are launched into a solid angle cone of 60 degrees; 

Trace the rays through the refractive index 3D mapped volume, and calculate 
the position of intersection and the phase of each of the rays at the entrance pupil wavefront sphere; 

Transform the converging wavefront in the entrance pupil of the optical 
system to a diverging wavefront at the exit pupil, and calculate the 3D point spread function 
corresponding to this wavefront using the Kirchhoff interference integral submenu; 

Output the 3D point spread function for the calculated location of the source 
point of the fan of rays; 

h) Run the decon_full executable file to perform the following: 

Input the location-dependent 3D point spread function and the fluorescent image for 
deconvolving, and run the iterative deconvolution. The program outputs the deconvoluted improved 
3D image. 

Additional technical descriptions of a preferred embodiment of the invention appear in 
-^t=yetT3ublishe d = articl^ D-and-arc incorporated herewith = 


It will be appreciated by persons skilled in the art that the present invention is not 
limited to what has been particularly shown and described hereinabove. Rather the scope of the 
present invention includes both combinations and subcombinations of the various features described 

16 



hereinabove as well as modifications and additions thereto which would occur to a person of skill in 
the art upon reading the foregoing description and which are not in the prior art. 
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.AO 

CLAIMS 


We claim: 

1. Apparatus for computational adaptive imaging including: 

an image information acquirer providing information relating to the refractive 
characteristics in a three-dimensional imaged volume; 

a ray tracer, utilizing the information relating to the refractive characteristics to trace 
a multiplicity of rays from a multiplicity of locations in the three-dimensional imaged volume through 
the three-dimensional imaged volume, thereby providing a location dependent point spread function; 
and 

a deconvolve^ utilizing the location dependent point spread function, to provide an 
output image corrected for distortions due to variations in the refractive characteristics in the 
three-dimensional imaged volume. 

2. Apparatus for computational adaptive imaging according to claim 1 and wherein the 
image information acquirer acquires at least two three-dimensional images of a three-dimensional 
imaged volume, at least one of the two three-dimensional images containing the information relating 
to the refractive characteristics in a three-dimensional imaged volume. 

3. Apparatus according to claim 2 and wherein the image acquirer acquires at least three 
three-dimensional images of the three-dimensional imaged volume. 

4. Apparatus according to claim 2 and wherein the image acquirer acquires a plurality of 
=4hfee*di mcnsional4mag es ofthe three - dimensionalimag e d volu m e , e ach th e ima g e having a discret e 


wavelength band. 
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5. Apparatus according to claim 2 and wherein the image acquirer acquires a multiplicity 

of three-dimensional images of the three-dimensional imaged volume, each the image having a 
wavelength band which is part of a continuum represented by the wavelength bands of the 
multiplicity of three-dimensional images. 


6. Apparatus according to any of claims 2-5 and wherein the ray tracer and the 
deconvolver utilize the information relating to the refractive characteristics in a three-dimensional 
imaged volume obtained from one of the three-dimensional images to correct at least another one of 
the three-dimensional images. 

7. Apparatus according to any of claims 1-6 and wherein the three-dimensional images 
are electromagnetic energy images. 

8. Apparatus according to claim 7 and wherein the three-dimensional images are infrared 
images. 

9. Apparatus according to any of claims 1-6 and wherein the three-dimensional images 
are non-electromagnetic images. 

10. Apparatus according to any of the preceding claims and wherein the image acquirer 
receives digital image data from a digital image source and derives therefrom the information relating 
to the refractive characteristics in a three-dimensional imaged volume. 

1 1 . Apparatus according to any of the preceding claims and wherein the image acquirer, 
the ray tracer and the deconvolver operate repeatedly over time to provide a multiplicity of output 
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images, each corrected fcd^Htortions due to variations in the re^^ve characteristics in the 
three-dimensional imaged volume. 

12. Apparatus according to any of the preceding claims and wherein the output image is 

an acoustic image and the refractive characteristics are characteristics of a material which the 
passage of acoustic energy therethrough. 


13. Apparatus according to any of the preceding claims and wherein the output image is 
an electromagnetic image and the refractive characteristics are characteristics of a material which the 
passage of electromagnetic energy therethrough. 

14. A method for computational adaptive imaging including the steps of: 

providing information relating to the refractive characteristics in a three-dimensional 
imaged volume; 

ray tracing, utilizing the information relating to the refractive characteristics, a 
multiplicity of rays from a multiplicity of locations in the three-dimensional imaged volume through 
the three-dimensional imaged volume, thereby providing a location dependent point spread function; 
and 

deconvolving, utilizing the location dependent point spread function, thereby 
providing an output image corrected for distortions due to variations in the refractive characteristics 
in the three-dimensional imaged volume. 

15 A ^me thod for computa ttoflafcad aptive imaging ac c or di ng to claim 14 and whereinJhe 

step of providing information includes acquiring at least two three-dimensional images of a 
three-dimensional imaged volume, at least one of the two three-dimensional images containing the 
information relating to the refractive characteristics in a three-dimensional imaged volume. 
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16. A method according to claim 15 and wherein the step of providing information 

includes acquiring at least three three-dimensional images of a three-dimensional imaged volume. 


17. A method according to claim 15 and wherein the step of providing information 
includes acquiring a plurality of three-dimensional images of the three-dimensional imaged volume, 
each the image having a discrete wavelength band. 

18. A method according to claim 15 and wherein the step of providing information 
includes acquiring a multiplicity of three-dimensional images of the three-dimensional imaged 
volume, each the image having a wavelength band which is part of a continuum represented by the 
wavelength bands of the multiplicity of three-dimensional images. 

19. A method according to any of claims 14 -18 and wherein the three-dimensional 
images are electromagnetic energy images. 

20. A method according to claim 19 and wherein the three-dimensional images are 
infrared images. 

21. A method according to any of claims 14-18 and wherein the three-dimensional 
images are non-electromagnetic images. 

77 A nu -lli Mi l ,i r.r.nrHi n o tn rrny-of th n pr nrpding claims 14 „ ? 1 ?nH ^ herein the Step of 

providing includes receiving digital image data from a digital image source and deriving therefrom 
the information relating to the refractive characteristics along a multiplicity of light paths in a 
three-dimensional imaged volume. 
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23. A method according to any of the preceding claims 14-22 and wherein the steps of 
providing information, ray tracing and deconvoluting operate repeatedly over time to provide a 
multiplicity of output images, each corrected for distortions due to variations in the refractive 
characteristics in the three-dimensional imaged volume. 

24. Apparatus for utilizing differential interference contrast images to provide 
three-dimensional refractive index information including a line integrator operating on differential 
interference contrast images displaying a directional derivative of refractive index of an object to 
invert the directional derivative thereof, thereby providing a plurality of two-dimensional 
representations of the refractive index of the object. 

25. Apparatus for utilizing differential interference contrast images to provide 
three-dimensional refractive index information according to claim 24 and also including a 
deconvolver performing deconvolution of the plurality of two-dimensional representations of the 
refractive index of the object, thereby reducing out-of-focus contributions to the two-dimensional 
representations of the refractive index of the object. 

26. A method for utilizing differential interference contrast images to provide 
three-dimensional refractive index information including performing line integration on differential 
interference contrast images displaying a directional derivative of refractive index of an object to 
invert the directional derivative thereof, thereby providing a plurality of two-dimensional 

" repres ent a t ions of the-refracti veindex of th e obj e ct. 


27. A method for utilizing differential interference contrast images to provide 

three-dimensional refractive index information according to claim 26 and also including performing 
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deconvolution of the plurality of two-dimensional representations of the refractive index of the 
object, thereby reducing out-of-focus contributions to the two-dimensional representations of the 
refractive index of the object. 

28. Apparatus for ray tracing through a medium having multiple variations in refractive 
index including: 

a computer employing an analytically determined path of a ray through the 
multiplicity of locations in the medium, for a plurality of rays impinging thereon in different 
directions, by utilizing known local variation of the refractive index at a multiplicity of locations in 
the medium. 

29. A method of ray tracing through a medium having multiple variations in refractive 
index including: 

determining local variation of the refractive index at a multiplicity of locations in the 

medium; 

analytically determining the path of a ray through the multiplicity of locations in the 
medium, for a plurality of rays impinging thereon in different directions. 

30. Apparatus for confocal microscopy including: 

a ray tracer, employing known variations of the refractive index in a 
three-dimensional sample for determining the paths of a multiplicity of rays emerging from at least 
one point in the sample and passing through the sample, thereby determining an aberrated wavefront 
frr n .i r h thf* po in t; nnri 

an adaptive optics controller utilizing the aberrated wavefront to control an adaptive 
optical element in a confocal microscope, thereby to correct aberrations resulting from the variations 
in the refractive index. 
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66 3B Dl DE 31 D3 84 33 80 36 53 13 A9 3D 93 70 .-.9 4S 31 2s. 15 DB 14 73 
OD 84 3A 32. C* ^ ^4 5C 84 E6 86 87 2C DC SA 9A 43 CC D6 37 91 9B 60 8F 
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PF C8 CF 93 DE 92 EE D3 .EE . EB D2 BF 4E 9D Al 31 33 E5 EC AO CD 32 15 5o 
5C 34 5C 3D 77 27 DE 6F 47 1C BO 18 36 31 19 EF 13 25 5E 19 C9 9C FA E8 
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45 37 8C 25 8A EE IS 5D A3 FC 4C 33 C9 F5 57 A3 34. DE F3 64 si 95 9D 31 
i 6 10 1!? p 3 i C 73 ii go 1 3 73 E3 10 E5 73 CF 39 53 3a ^3 C4 9D =B -2 

B8 F5 *E2 3A 90 50 77 ID C3 9C E7 F3 DB 3A 13 E3 43 11 36 C7 8D 43 03 29 
C5 50 SF 49 30 06 A6 F2 F9 6D 36 31 83 C6 91 jA 8/ 44 ,1 62 43 CX DC 33 
C4 Bl 51 PA D° 35 F2 D9 C Fl 1C FD 43 S2 43 F3 SS C4 37 10 13 /D -8 53 
FA IE 71 23 9DE5 66 E2 DC 7F 28 32 84 FC EF 64 05 03 BB D5 96 FC 32 E2 
AS 2F 41 8E FF 91 74 F7 3F 89 S4 73 F8 FB DF 09 43 SA 42 24 FF 2F /4 *r 
5P C4 34 AD 30 Bl B5 3F 4F 24 33 E9 9C 94 13 82 FS EC /9 9o 7D 33 CI 63 
: 9 3 SB OF 89 31 5F OA 31 FS 03 DC 73 47 64 53 72 49 C4 2F F7 E7 3F D3 « 
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57 S3 02 1A 71 2? 33 DA 07 2A 11 ?0 32 32 30 C5 3C 4r D9 32 73 A3 
91 ~1 -15 C? CS AO D3 6D 95 05 1A 59 31 4/ 30 37 -Z 2- FO iZ z, ox r3 l-x 
-•d ." B '-- =2 63 C3 47 69 OS A9 49 10 36 OA' 51 AC FO 23 33 35. 51 40 Oa 
35 ^0 7^ =3 47 33 D5 20 3D C9 69 35 33 21 33 F9 05 3D 27 14 C4. DD 63 C2 
•r 63 l~ Z* 7- 3F 33 46 67 4C 22 FD D7 CA C3 61 D4 55 50 99 =o 5/ Fl DD 
Yn r 3 £:. E5 34 c A iA i3 9? Co F3 09 4F 79 5C 23 FA 4.- 14 A3 42 3D 3C 
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05 3F F5 5D S3 36 F3 23 AA C7 AS 24 ' CS 2F 90 CD 47 02 2u Z= =3 a4 3o r.| 
93 25 32 DA 49 F3 37 12 33 C5 53 DA 22 27 20 63 46 73 5D F3 4* b* r» ^3 
~D F0 D9 3C 33 S3 59 42 73 6C 4D 7D 93 05 CF 23 .-2 AC - .- 1 - - =.5 ^ 
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27 66 79 32 0D 9F 01 57 El FS 2E 37 D7 3 9 EC 91 53 ID aA oA r/ D9 6n /u 
21 17 D3 EA A2 B3 65 FF OS EF 9F C5 F3 51 BC IF 31 33 /* A4 oD 27 9F =1 
±1 53 CF 6D 32 7C 0B FB 83 20 S3 =3 31 DF 60 FF 61 ol 4D =.* -A t9 .0 ^ 
2C 3 A 7C 0 8 9D 5 5 5 5 3 9 B7 8F SC 33 A7 F2 SS 44 F2 D7 AO FD -A ,2 C9 2^ 
C9 F6 A3 39 89 C7 SO 63 9C B3 C4 C4 21-8S,-F4 31 7C 02 ^ Oo CI ro ,f 

55 9' 13 43 C2 8F 0C A7 CE 8D 9D 30 .OF" 13 41 19 30 Ez c4 oo ^C *6 0b 

C5 31 T1 C4 37 IC 77 Bl CD 46 07 31 33 D6 12 AF CA 98 86 D9 53 20 4A 03 
D3 8A 25 F4 DF 50 51 BF C7 50 37 33 30 95 BB 61 23 7B 3C 2a jA A3 7A 06 
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